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(57) Abstract: The preierrcd embodiment addresses the problem of delivering stereoscopic media in electronic form (images, videos, 
animations, object models, etc.). First, it provides a single format with independent right and left channels to represent the stereo- 
scopic media. Second, it provides a means of displaying stereoscopic media inside a movable windowed area while eliminating 
pseudostereo conditions during movement. Third, it provides automatic and manual optimization adjustments (parallax shift adjust- 
ment, brightness control, color adjustment, and cross-talk reduction) to the stereoscopic media based on viewing hardware, monitor 

£^ size, and media content for optimal viewing quality. Fourth, it provides seamless support for monoscopic (2D) viewing modes al- 

^ lowing delivery of said stereoscopic media in a normal 2D viewing mode. 
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ELECTRONIC STEREOSCOPIC MEDIA DEETVERY SYSTEM 
TECHNICAL FIELD 

The invention is directed to providing a means of delivering stereoscopic media on and off the 
Internet. In particular the invention provides a means of encoding stereoscopic media, transmitting and 
5 storing stereoscopic media, displaying stereoscopic media, and providing helpful tools for viewing the 
media. 

BACKGROUND OF THE INVENTION 

Stereoscopic images have been in use for hundreds of years. Recently stereoscopic images, and 
other forms of stereoscopic media like animations and video, have been converted to electronic form 

10 for display on personal computers, the Internet, and on other electronic media like CD-ROMs. 
Stereoscopic media has been successfully used in numerous applications ranging from medical 
imaging, to entertainment, to training, to electronic commerce. 

Stereoscopic media and viewing systems can take on numerous formats. For example, there are 
several ways to encode a still stereoscopic 3D image including red/blue anaglyphic format, side-by- 

15 side, interleaved or line-alternate formats, etc. There are also numerous viewing systems available to 
viewing stereoscopic media: red/blue glasses, active shutter glasses, high-speed page-flipping graphics 
cards with shutter glasses, line-blanking viewing systems, cross-eye lens systems, etc. Many of these 
media and viewing formats are not compatible with each other. The fact that the stereoscopic media 
for these various systems is not compatible makes it difficult for a content developer, such as a CD- 

20 ROM developer, or Website developer, to support all of the stereoscopic viewing' devices on the 
market. 

Several patents contain background information for this disclosure. They include: US Patent 
6,028,649 dated 02/22/2000 and entitled "Image Display Systems having Direct and Projection 

l 
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Polarized Spatially Multiplexed Images of 3-D Objects for Use in Stereoscopically Viewing The Same 
With High Image Quality and Resolution. 

In order to facilitate the growth of stereoscopic 3D media in electronic form, an easy to use, 
multi-format delivery system is needed This invention presents new stereoscopic media delivery 
5 system that includes means for encoding stereoscopic media, transmitting and storing stereoscopic 
media, displaying stereoscopic media, and providing helpful tools for viewing the media. 
SUMMARY OF THE INVENTION 

The preferred embodiment addresses the problem of delivering stereoscopic media in electronic 
form (images, videos, animations, object models, etc.). Firstly, it provides a single format with 
10. independent right and left channels (with an option for mixed or combined right and left channels) to 
represent the stereoscopic media. Secondly, it provides a means of displaying stereoscopic media 
inside a movable windowed area while eliminating pseudostereo conditions during movement. 
Thirdly, it provides automatic and manual optimization adjustments such as parallax shift adjustment, 
brightness control, color adjustment, and cross-talk reduction to the stereoscopic media based on 
.5 viewing hardware, monitor size, and media content for optimal viewing quality. Fourthly, it provides 
seamless support for monoscopic (2D) viewing modes allowing delivery of said stereoscopic media in 
a normal 2D viewing mode. 

These and other features of the present invention will be apparent from the following 
description of the drawings, detailed description, and appended claims and drawings, 
to BRIEF DESCRIPTION OF THE DRAWINGS 

Figure 1 illustrates the various display formats embodied in the invention; 
Figure 2 illustrates the contents of a window displayed in stereo; 
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DETAILED DESCRIPTION OF THEMNVENTION 

The embodiments of the invention include display methods, encoding methods and tools. 
With regard to display methods this includes a single media file format that is converted to various 
display formats on the user side; stereoscopic media in a window such as a browser or application; 
5 stereoscopic preservation in a window during scrolling and window movement; support of auto- 
detection 3D stereo hardware systems; script buttons (VRR scripts) to change global stereo formats; 
stereo media file formats that contain sub media such as VRR and blocks; parallax shift adjustments 
based on physical size of display window; automatic brightness adjustments; color ' 
calibration/adjustments for physical 3D viewing mechanisms, including variations in display devices; 
10 crosstalk reduction techniques on user side; smart stereo scaling; integration of stereo media types into 
one viewer with script interaction; monoscopic and stereoscopic viewing that , allows greater 
distribution since both types can be viewed within one system; save and conversion of one format into 
another from the Internet using a local drive from the original source; automatic free view image size 
adjustment to minimize viewing fatigue; pseudostereo correction based on image processing of a few 
:5 lines or the entire image; scaling stereo media, so that the left and right sources are preserved; and 
improvements to Anaglyph display methods. Since the format of the original left and right is known, 
as designated by the tag within the Stereoscopic 3D Media file, the scaling can be done while 
preserving stereo. Additionally, looking at the storage method used, it is necessary to take the 
appropriate actions to scale the media while preserving the stereo arid to perform scaling done to 
0 increase or decrease the display size of the stereoscopic media. 

In existing systems, stereo media can only be viewed at the original size. The embodiment of 
Figure 1 provides a mechanism to increase or decrease the size at which the media is displayed while 
preserving the stereo. As shown in Figure 1 with analysis 10, the left and right media are extracted 
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current vertical location of-the window, and then places the first line of the stereo image on an even 
scan line. Whenever a user scrolls within that window, or moves the window itself, the application 
checks the odd/even status, and adjusts accordingly. In the case of vertically scrolling within a window 
or if the window itself is moved, the first line of the stereo image is placed on an even scan line. When 
the image is scrolled vertically, movement is done in even increments so as to maintain stereo or the 
right and left images are swapped for each pixel move to maintain stereo. For the case of horizontal 
scrolling, the same methods as vertical scrolling apply. 

To display interleaved stereo images in a window or on a full-screen, the left and right fields or views 
must be preserved and presented consistently. For example the first horizontal line of a stereo image 
that is presented within a window may represent the right field or view and the second horizontal line 
the left field or view. In this example, the first horizontal line of the display device is e>7fcn, or 
represented by a zero, the right field/view is to be presented on even scan lines, and the left field/view 
is to be presented on odd scan lines to maintain stereo. If the left field/view were presented on the even 
scan lines and the right field/view on the odd lines, then the result would be pseudostereo. The 
selection of associating the right field/view with even scan lines is arbitrary — the reverse can also be 
true provided consistency is maintained. The left field/view may be on the odd scan lines, the right 
field/view on the even. 

The exemplary embodiment preserves left field/view and right field/view in a several ways. A 
first method is to snap the window to an even scan line, which ensures that the window starts on an 
even horizontal scan line. Another method is to look at the even/oddness of the first horizontal line 
where the stereo media starts. If the right field/view is on the even scan line, and the left field/view is 
on the odd, then no action is required, otherwise the left/right fields/views can be swapped — placing 
the left content on the even lines, and the right content on odd, for every horizontal line in the media, 
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details of a-Hne-interleaved stereo image. The even line of the stereo iffiSge~is not aligned with an even 
scan line of the monitor, which results in the display of a pseudoscopic, or reverse image. 

Figure 4 illustrates the contents of a scrolling window displayed in stereo 300. A stereo image 
is presented on a computer screen in a window 302. This particular window scrolls vertically. The 
5 first line of the image is aligned on an even scan line 304 with the monitor, which allows for 
stereoscopic presentation. Whenever vertical scrolling occurs, even increments are made so that the 
image is always presented in stereo. 

Figure 5 illustrates a General Software Flowchart 400. Upon start 402, the application checks to 
see where the image starts, on an even or odd scan line 404. If on an even scan line 406, a stereo image 
0 is then displayed 408. If the image starts on an odd scan line 410, then the image is shifted to an even 
scan line or the left and right field/views are swapped and then displayed 442. Vertical scrolling begins 
418 and is done by an even increment 420. 

To scale stereo media, the left and right source must be preserved. Since the format of the 
original left and right is known, as designated by the tag within the Stereoscopic 3D Media file, scaling 
5 can be done while preserving stereo. An embodiment ooks at the storage method used, then takes the 
appropriate actions to scale the media while preserving the stereo. Scaling may done to increase or 
decrease the display size of the stereoscopic media. 

Figure 6 illustrates an analysis of the left and right media 500 that are extracted from the 
Stereoscopic 3D Media file 502, individually scaled504, 506, recombined into the selected display 508, 
• and then the resulting scaled Stereoscopic Media is displayed 510. 

To scale stereo media, the left and right source must be preserved. Since the format of the 
original left and right is known, as designated by the tag within the Stereoscopic 3D Media file, the 
scaling can be done while preserving stereo. The system will look at the storage method used, and then 
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This is accomplished by showing either the left onight mono image. The user can select 
whether to view the left or right monoscopic view. Users without a physical stereo viewing device can 
see the media in monoscopic form by selecting to use either the right or left monoscopic views. 

An embodiment of the invention contains automatic adjustment of Brightness/Contrast/Image 
properties adjustment based on viewing mechanism. No prior Electronic viewing system adjusts media 
brightness/contrast based upon the display method. This embodiment adjusts the final display of the 
stereo media to accommodate LC shutter-glasses, and LC shutter-glasses with line-blanker type 
products. When stereo media is viewed.through an LC shutter-glass viewing system it appears darker 
due to the shuttering system. Additionally when stereo media is viewed through an LC Shutter-glass 
coupled with a hue blanker, the media appears even darker. This embodiment will adjust the 
brightness, contrast and other mediaproperties to compensate for the color distortion/darkQess incurred 
by the viewing mechanism. Figure 8 illustrates the automatic adjustment of Brightness/Contrast/Image 
properties adjustment based on viewing mechanism 700. The Stereoscopic Media file 702 is queried 
arid is displayed using shutter glasses 704. The Brightness adjustment system 706 adjusts the 
brightness of the media file to accommodate the physical viewing mechanism 708. This same 
embodiment can compensate for uneven image luminance between the left and right eyes when using 
anaglyphic colored viewing glasses. 

Algorithms are used to reduce crosstalk between the left and right views. Other stereoscopic 
viewing systems do not incorporate a system to reduce crosstalk. Crosstalk often appears in stereo 
media and can be referred to as ghosting. Bright in one eye, dark in the other, occupying the same point 
in the scene can create crosstalk. Areas of high contrast in stereo media are subject to crosstalk. This 
ghosting occurs because each eye is seeing some of the media intended only for the other eye, and in 
this case there may be very bright content in one eye, and darker content in the other eye. 
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There is an embodiment that saves and converts one format into another from the Internet using 
a local drive from original source. The viewing system takes a Stereoscopic Media File, displays it on 
the user side according to the user's display preferences, and saves a local copy in whatever display 
format the user selects. The system can convert the Stereoscopic Media into the display method 
selected by the user, and save the result on the user's local drive. Alternately, the original form of the 
Stereoscopic 3D Media file may be saved on the user's local drive. This embodiment makes it possible 
to support special 3D formats that are not easy to generate in real time. 

A color caUbration/adjustment for physical 3D viewing mechanisms, compensates for 
variations in the display devices. An embodiment includes the functionality to adjust stereo media 
properties, thereby preserving the original color, hues, saturation, etc. when viewed through a physical 
viewing mechanism. For example, when looking through liquid crystal shutter-glass systems, the LC 
may introduce additional yellow coloring to the subject matter. Based on the user's view settings that 
tell the originator what kind of viewing device the user may be using. Another embodiment of the 
invention corrects for distortions introduced by the viewing mechanism. Another example could be 
for anaglyph viewing. For this example, the viewing system goes through a color calibration to attempt 
to more closely match the coloring of the anaglyph lenses with the coloring of the monitor or display 
device. 

A stereo media file format may contain certain sub media such as VRR and blocks. An 
embodiment supports a stereoscopic media file that contains sub-media. Specifically, a file structure is 
created to store and preserve various types of stereo media in various formats. Additionally, this file 
format can also store monoscopic media, as well as audio or other data. This one file format can store 
multiple or single stereo/non-stereo media elements. Figure 10 illustrates a VRR file 1000 that may 
contain a script 1002, a Stereo Still Image 1004, a Stereoscopic Animation/movie 1006, Stereoscopic 
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Another embodiment automatically (or manually if desired) adjusts with overaD left" and right 
image shift to compensate for image magnification. When a 3D stereoscopic image is enlarged and 
displayed on a viewing system that has a larger image size then the target system, there is a potential of 
creating large separations between objects in the 3D stereoscopic image that can lead to eyestrain for 
the viewer. The present invention stores important parameters about the 3D stereoscopic image like 
width, height, target screen size, etc. When the 3D stereoscopic image is to be displayed on a display 
that is larger or smaller than the target screen size, then the 3D stereoscopic image is adjusted 
accordingly to minimize eye fatigue for the user. 

One embodiment utilizes image processing to detect pseudo-stereo 3D stereoscopic content. It 
is common for content authors to sometimes reverse the right and left eyes when creating 3D 
stereoscope content. When this happens, the display system will pTesenrthe wrong image to the 
viewer's eyes. This embodiment attempts to correct this problem by comparing portions of the right 
and left image content to determine if a pseudo condition exists and then swaps the right and left 
images to correct for the problem. 

Another embodiment encodes the full-color left and right images in separate channels and 
compresses the left and right channels independently. This technique provides less compression 
artifacts and reduces crosstalk when compared to anaglyphic storage techniques. In comparison, 
analgvph storage techniques, which are widely used on the Internet, combine the right and left images 
in separate color channels of a single image and then compressed the resulting image. This technique 
results in the introduction of crosstalk because most compression techniques, like JPEG that is 
commonly used, reduce the color space of an image drastically, which in turn compromises the quality 
of the Anaglyphic storage technique. 
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One embodiment-stores a series of 3D stereoscopic images of an object into^ file. Figure 16 
illustrates one way to store a series of N images into a single resource file. The first image 1600 is at 
the top of the file and the rest of the images follow sequentially until the last file 1602. The collection 
of these images forms an object movie data file 1604. Figure 17 shows the relationship between the 
images and the object in question 1700. As the figure illustrates, the point of view of the camera is 
swung around the object to generate all of the intermediate images. When these images are displayed 
in a sequential fashion, it appears that the object is rotating on the screen in 3D depth. When the 
images are displayed in reverse order, the object appears to turn in the opposite direction. If the 
playback of the images is linked to the right/left movement of a mouse point on the screen, it will 
appear that the user is actually rotating the object on the screen. This example illustrates a one 
dimensional object movie. It is possible to store additional sets^of rotational images sets to simulate 
other views of the object (example: doors open or closed on a car model). 

Another embodiment allows a larger 3D stereoscopic image or panorama to be scrolled within a 
smaller viewing window as shown in Figure 18. The viewing window 1802 is a fixed size and does 
not move with respect to the viewing screen. The 3D stereoscopic image 1800 is larger than the 
viewing window and therefore must be scrolled horizontally and vertically within the viewing window 
in order for the viewer to see the entire image. This is always the case with panoramic images that 
contain a very wide or tall view of a scene. When the image is panned within the viewing window, the 
invention ensures that the left and right views are presented to the correct eyes. For example, in the 
row interleaved viewing method, if the larger 3D stereoscopic image is moved vertically by a single 
pixel, the image will flip into pseudostereo mode. The present invention will either shift the image an 
additional line to make it viewable in non-pseudostereo or it will flip the left and right images to 
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morphing -techniques to synthesize a new right and left image that~arer closer together. The same 
technique can be used to simulate a wider stereoscopic camera separation to increase the depth effect. 
The amount of adjustment can be selected by the user to suit their viewing condition and capabilities. 
Figure 19 illustrates an original right 1902 and left 1904 camera views of an object 1900. The new 
5 synthesized right 1906 and left 1908 views will be easier to view for some users. 

Another embodiment of the invention provides an automatic alignment and/or re-alignment of 
the left and right images. Many 3D stereoscopic images are created incorrectly and may have vertical, / 
horizontal, or rotational misalignment that will cause eyestrain for the viewer. Using image correlation 
10 techniques, horizontal and vertical misalignment of the right and left views is corrected. Further 
correlation is applied to correct for rotational alignment issues. 

Another embodiment provides a tool that compares the left and right images to determine if 
stereoscopic information is lost during compression. Compression is used to reduce the size of a 
15 stereoscopic image so that it can be easily transmitted over a low bandwidth connection. If too much 
compression is applied, the image will loose its stereoscopic impact. This embodiment provides a 
measure of the quality of the stereoscopic image that can be used to readjust the compression system. 

Another embodiment allows both local content, residing on the users workstation, and remote 
20 content, residing on a server or web site to be viewed using the display methods. This embodiment is 
needed to allow for remote streaming playback as well as to allow for local playback of files that are 
delivered via cd-rom or other storage means or for content that is downloaded from a remote location 
to a local storage location before being displayed. 
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-View 2 can be combined to form a left right pair (respectively). The next 3D stereoscopic view is 
formed by combining view 2 with view 3 (left and right respectively). Each image in the original 
object movie is used twice. If the original 2D object movie contains enough images (60 images 
produces good results) then the converted 3D object movie will be usable. If there are not enough 
images, less than 40, then better results are achieved if the previously mentioned embodiment of 
generating intermediate views using image interpolation can be applied to the object movie conversion. 
Using image interpolation allows intermediate stereoscopic views to be generated, which have much 
less camera separation and are easier to view. 



10 The present invention can also be embodied in the form of computer program code, for 

example, whether stored in a storage medium, loadecLinto and/or executed by a computer, or 
transmitted over some transmission medium, such as over electrical wiring or cabling, through fiber 
optics, via electromagnetic radiation or via the Internet, wherein, when the computer program code is 
loaded into and executed by a computer, the computer becomes an apparatus for practicing the 

15 invention. When implemented on a general-purpose microprocessor, the computer program code 
segments configure the microprocessor to create specific logic circuits. 

The modifications to the various aspects of the present invention described above are 
merely exemplary. It is understood that other modifications to the illustrative embodiments will readily 
occur to persons with ordinary skill in the art. All such modifications and variations are deemed to be 
20 within the scope and spirit of the present invention as defined by the accompanying Claims. 



21 



) <WO 0197531A2_I_> 



' WO 01/97531 PCT/US01/18873 

• • 

7. The method of claim 1 further-comprising stereoscopic preservation in a window during 
scrolling and window movement. 

8. The method of claim 1 comprising supporting of auto-detection 3D stereo hardware 
systems. 

5 . 9. The method of claim 1 comprising script buttons (VRR scripts) for changing global 

stereo formats. 

10. The method of claim 9 further comprising stereoscopic media file formats that contain 
sub media such as VRR and blocks. 

1 1 - The method of claim lcomprising parallax shift adjustments based on a physical size of 
10 display window. 

12. The method of claim 1 further comprising automatic brightness adjustments. 

13. The method of claim 1 further comprising color calibration/adjustments for physical 3D 
viewing mechanisms, including variations in display devices. 

14. The method of claim 1 further comprising crosstalk reduction techniques on a user side. 

15. The method of claim 1 further comprising smart stereo scaling. 

16. The method of claim 1 further comprising integration of stereo media types into one 
viewer with script interaction. 

17. The method of claim 1 further comprising monoscopic and stereoscopic viewing that 
allows greater distribution since both types can be viewed within one system. 

18. The method of claim 1 comprising saving and converting one format into another from 
the Internet using a local drive from an original source. 

19. The method of claim 1 further comprising automatic free view image size adjustment to 
minimize viewing fatigue. 
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30. The method of claim-29 including using stereoscopic panning that preserve^ereo 
image alignment. 31. The method of claim 30 uses a co-existence of Java software and a plug- 
in solution to minimize downloading. 

32. The method of claim 31 using image interpolation to generate in between stereo views 
to minimize or maximize the stereo separation 

33. , The method of claim 32 using image interpolation for converting a 2D object movie to a 
3D stereoscopic object display and provide a background download capability. 
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